🌴KapoeiraπŸ•ΊπŸ’ƒ

kapoeira

Presentation

kara
Mehdi Rebiai

Presentation

odile
Johanna Vauchel

🎬 Kapoeira story πŸ“½οΈ

Enrich and collect data

enrichData

We are perfect

perfect

NO !

TODO schΓ©ma si bug ici, bug dans les streams suivants avec mermaid

Solution ?

TESTS !

How to test ?

TopologyTestDriver

fast

Fast and efficient…​

…​ But

fail

it did not test the integration with the Kafka cluster

…​ But

how to test several streams ?

(individually OK but together KO)

…​ But it’s not for QA

les nuls police

Listen, let’s DEV do their job !

…​ But it’s code

simon kara

how to communicate with DEV/PO/QA ?

What do we need ?

Integration tests with simple syntax

Inspiration : Karate

karate

  • HTTP-based APIs

  • Peter Thomas - 2017

  • simple syntax (Gherkin)

Birth of Kapoeira

Kapoeira πŸ€” ?

Integration Tests

test pyramid

Kafka Stream

kafkalogo

2020

  • Inner tool

  • console-script confluent

202x ZIO

zio

2023 Open Source

kapoeira logo text

Inside Kapoeira

Cucumber Scala, using specific Gherkin DSL.

cucumber

How does it work ?

kapoeira diagram

Example

Feature: upper-case
  Background:
    Given input topic
      | topic              | alias    | key_type | value_type |
      | topic-simple-value | topic_in | string   | string     | (1)

    And output topic
      | topic                   | alias            | key_type | value_type | readTimeoutInSecond |
      | topic-simple-value      | topic_string_out | string   | string     | 5                   |
      | topic-upper-case-string | topic_out        | string   | string     | 5                   |  (2)
    And var myKey = call function: uuid

  Scenario: My first scenario
    When records with key and value are sent (3)
      | topic_alias | key      | value |
      | topic_in    | ${myKey} | a     |
      | topic_in    | ${myKey} | b     |
      | topic_in    | ${myKey} | c     |
    Then expected records                    (4)
      | topic_alias      | key      | value    |
      | topic_string_out | ${myKey} | input_1  |
      | topic_string_out | ${myKey} | input_2  |
      | topic_string_out | ${myKey} | input_3  |
      | topic_out        | ${myKey} | result_1 |
      | topic_out        | ${myKey} | result_2 |
      | topic_out        | ${myKey} | result_3 |
    And assert input_1 $ == "a"             (5)
    And assert input_2 $ == "b"
    And assert input_3 $ == "c"

    And assert result_1 $ == "A"
    And assert result_2 $ == "B"
    And assert result_3 $ == "C"

Report

report

REX

Advantages

advantages

Advantages

  • test infra

  • simple to use

  • tool to communicate

  • documentation

  • acceptance tests

How to use ?

banco

How to build ?

docker build -t kapoeira:latest .

How to use ?

Docker Hub image

docker run --rm -ti \
-v <PATH_TO_YOUR_FEATURES_FOLDER>:/features \
-v /var/run/docker.sock:/var/run/docker.sock \
-e KAFKA_BOOTSTRAP_SERVER=<HOST:PORT[,HOST2:PORT2,HOST3:PORT3,...]> \
-e KAFKA_SCHEMA_REGISTRY_URL=<URL> \
-e KAFKA_USER=<XXX> \
-e KAFKA_PASSWORD=<****> \
-e JAAS_AUTHENT=<true (default) | false> \
-e LOGGING_LEVEL=<INFO (default) | ERROR | ...> \
-e THREADS=<8 (default) | ... > \
lectratech/kapoeira

How to contribute ?

TODO

DΓ©mo

buger quiz

Architecture

burger quiz

Docker commands

docker compose build --no-cache
docker compose up -d
docker restart kapoeira

burger.feature

Feature: Burger πŸ” feature

  Background:
    Given input topic
      | topic     | alias        | key_type | value_type |
      | bread     | bread-in     | string   | string     |
      | vegetable | vegetable-in | string   | string     |
      | meat      | meat-in      | string   | string     |
    And output topic
      | topic  | alias      | key_type | value_type | readTimeoutInSecond |
      | burger | burger-out | string   | string     | 5                   |
    And var uuid = call function: uuid

  Scenario: Nominal
    When records with key and value are sent
      | topic_alias  | key        | value |
      | bread-in     | 🀀_${uuid} | 🍞    |
      | vegetable-in | 🀀_${uuid} | πŸ…    |
      | meat-in      | 🀀_${uuid} | πŸ₯©    |
    Then expected records
      | topic_alias | key        | value |
      | burger-out  | 🀀_${uuid} | order |
    And assert order $ == "πŸ”"

  Scenario: Not a burger
    When records with key and value are sent
      | topic_alias  | key        | value |
      | bread-in     | 🀀_${uuid} | 🍞    |
      | vegetable-in | 🀀_${uuid} | πŸ₯•    |
      | meat-in      | 🀀_${uuid} | πŸ₯©    |
    Then expected records
      | topic_alias | key        | value |
      | burger-out  | 🀀_${uuid} | order |
    And assert order $ == "🍞 + πŸ₯• + πŸ₯©"

  Scenario Outline: Many customers
    When records with key and value are sent
      | topic_alias  | key            | value       |
      | bread-in     | <user>_${uuid} | <bread>     |
      | vegetable-in | <user>_${uuid} | <vegetable> |
      | meat-in      | <user>_${uuid} | <meat>      |
    Then expected records
      | topic_alias | key            | value |
      | burger-out  | <user>_${uuid} | order |
    And assert order $ == "<result>"

    Examples:
      | user | bread | vegetable | meat | result |
      | 🀀   | 🍞    | πŸ…        | πŸ₯©   | πŸ”     |
      | πŸ˜‹   | 🍞    | πŸ…        | πŸ—   | πŸ”     |
      | 😑   | 🍞    | πŸ…        | 🐟   | πŸ”     |

meal.feature

Feature: Meal πŸ› feature

  Background:
    Given input topic
      | topic       | alias          | key_type | value_type |
      | bread       | bread-in       | string   | string     |
      | vegetable   | vegetable-in   | string   | string     |
      | meat        | meat-in        | string   | string     |
      | side-dishes | side-dishes-in | string   | string     |
    And output topic
      | topic | alias    | key_type | value_type | readTimeoutInSecond |
      | meal  | meal-out | string   | string     | 20                  |
    And var uuid = call function: uuid

  Scenario: Left Join with Left first
    When records with key and value are sent
      | topic_alias    | key        | value |
      | bread-in       | 🀀_${uuid} | 🍞    |
      | vegetable-in   | 🀀_${uuid} | πŸ…    |
      | meat-in        | 🀀_${uuid} | πŸ₯©    |
      | side-dishes-in | 🀀_${uuid} | πŸ₯”πŸΊ  |
    Then expected records
      | topic_alias | key        | value |
      | meal-out    | 🀀_${uuid} | notif |
      | meal-out    | 🀀_${uuid} | order |
    And assert notif $ == "πŸ”"
    And assert order $ == "πŸ›(πŸ” + 🍟🍺)"

  Scenario: Left Join with Right first
    When records with key and value are sent
      | topic_alias    | key        | value |
      | side-dishes-in | 🀀_${uuid} | πŸ₯”πŸ·  |
      | bread-in       | 🀀_${uuid} | 🍞    |
      | vegetable-in   | 🀀_${uuid} | πŸ…    |
      | meat-in        | 🀀_${uuid} | πŸ₯©    |
    Then expected records
      | topic_alias | key        | value |
      | meal-out    | 🀀_${uuid} | order |
    And assert order $ == "πŸ›(πŸ” + 🍟🍷)"